Breakthrough technology has emerged which allows the navigation of a catheter tip through a tortuous channel, such as those found in the pulmonary system, to a predetermined target. This technology compares the real-time movement of a sensor against a three-dimensional digital map of the targeted area of the body (for purposes of explanation, the pulmonary airways of the lungs will be used hereinafter, though one skilled in the art will realize the present invention could be used in any body cavity or system: circulatory, digestive, pulmonary, to name a few).
Such technology is described in U.S. Pat. Nos. 6,188,355; 6,226,543; 6,558,333; 6,574,498; 6,593,884; 6,615,155; 6,702,780; 6,711,429; 6,833,814; 6,947,788; and 6,996,430, all to Gilboa or Gilboa et al.; and U.S. Published Applications Pub. Nos. 2002/0193686; 2003/0074011; 2003/0216639; 2004/0249267 to either Gilboa or Gilboa et al. All of these references are incorporated herein in their entireties.
Using this technology begins with recording a plurality of images of the applicable portion of the patient, for example, the lungs. These images are often recorded using CT technology. CT images are two-dimensional slices of a portion of the patient. After taking several, parallel images, the images may be “assembled” by a computer to form a three-dimensional model, or “CT volume” of the lungs.
The CT volume is used during the procedure as a map to the target. The physician navigates a steerable probe that has a trackable sensor at its distal tip. The sensor provides the system with a real-time image of its location. However, because the image of the sensor location appears as a vector on the screen, the image has no context without superimposing the CT volume over the image provided by the sensor. The act of superimposing the CT volume and the sensor image is known as “registration.”
Sensor Probe-Based Registration Methods
There are various registration methods, some of which are described in the aforementioned references, and utilize a probe with a trackable sensor, as described above. For example, point registration involves selecting a plurality of points, typically identifiable anatomical landmarks, inside the lung from the CT volume and then using the sensor (with the help of an endoscope) and “clicking” on each of the corresponding landmarks in the lung. Clicking on the landmarks refers to activating a record feature on the sensor that signifies the registration point should be recorded. The recorded points are then aligned with the points in the CT volume, such that registration is achieved. This method works well for initial registration in the central area but as the sensor is navigated to the distal portions of the lungs, the registration becomes less accurate as the distal airways are smaller. Also, the point registration method matches a “snapshot” location of the landmarks to another “snapshot” (CT volume) of the lungs. Each snapshot is taken at different times and, potentially, at different points in the breathing cycle. Due to the dynamic nature of the lungs, the shape of the lungs during the CT scan is likely not the same as the shape of those same lungs during the procedure. Moreover, because the physician is “clicking” on the landmarks over the course of several breathing cycles, it is up to the physician to approximate the timing of his clicking so that it roughly matches the point in the breathing cycle at which the CT scan was taken. This leaves much room for error. Finally, it is time consuming for the physician to maneuver the sensor tip to the various landmarks.
Another example of a registration method utilizing a trackable sensor involves recording a segment of an airway and shape-match that segment to a corresponding segment in the CT volume. This method of registration suffers similar setbacks to the point registration method, though it can be used in more distal airways because an endoscope is not required. The registration should be conducted more than once to keep the registration updated. It may be inconvenient or otherwise undesirable to require additional registration steps from a physician. Additionally, this method requires that a good image exists in the CT volume for any given airway occupied by the sensor. If for example, the CT scan resulted in an airway shadowed by a blood vessel, for example, the registration will suffer because the shape data on that airway is compromised.
Another registration method tailored for trackable sensors is known as “Adaptive Navigation” and was developed and described in U.S. Published Application 2008/0118135 to Averbuch et al., incorporated by reference herein in its entirety. This registration technique operates on the assumption that the sensor remains in the airways at all times. The position of the sensor is recorded as the sensor is advanced, thus providing a shaped historical path of where the sensor has been. This registration method requires the development of a computer-generated and automatically or manually segmented “Bronchial Tree” (BT). The shape of the historical path is matched to a corresponding shape in the BT.
Segmenting the BT involves converting the CT volume into a series of digitally-identified branches to develop, or “grow,” a virtual model of the lungs. Automatic segmentation works well on the well-defined, larger airways and smaller airways that were imaged well in the CT scans. However, as the airways get smaller, the CT scan gets “noisier” and makes continued automatic segmentation inaccurate. Noise results from poor image quality, small airways, or airways that are shadowed by other features such as blood vessels. Noise can cause the automatic segmentation process to generate false branches and/or loops—airways that rejoin, an occurrence not found in the actual lungs.
Another registration method is herein referred to as “feature-based registration.” When the CT scans are taken, the CT machine records each image as a plurality of pixels. When the various scans are assembled together to form a CT volume, voxels (volumetric pixels) appear and can be defined as volume elements, representing values on a regular grid in three dimensional space. Each of the voxels is assigned a number based on the tissue density Housefield number. This density value can be associated with gray level or color using well known window-leveling techniques.
The sensing volume of the electromagnetic field of the sensor system is also voxelized by digitizing it into voxels of a specific size compatible with the CT volume. Each voxel visited by the sensor can be assigned a value that correlates to the frequency with which that voxel is visited by the sensor. The densities of the voxels in the CT volume are adjusted according to these values, thereby creating clouds of voxels in the CT volume having varying densities. These voxels clouds or clusters thus match the interior anatomical features of the lungs.
By using a voxel-based approach, registration is actually accomplished by comparing anatomical cavity features to cavity voxels, as opposed to anatomical shapes or locations to structure shapes or locations. An advantage of this approach is that air-filled cavities are of a predictable range of densities.
Image-Based Registration Methods
Some registration methods are used with systems that use a bronchoscope without a trackable sensor. One of these registration methods compares an image taken by a video camera to a virtual model of the airways. The virtual model includes surfaces, reflections and shadows. This method while herein be referred to as “virtual surface matching.” A virtual camera is established to generate a viewpoint and a virtual light source is used to provide the reflections, shadows, and surface texture. The virtual camera and light source are matched to the actual video camera and light source so that an “apples to apples” comparison can be performed. Essentially, the virtual model is a library of thousands of computer-generated images of the lungs, from various viewpoints. Hence, the image taken by the video camera is compared against this large library, in the same way a fingerprint is lifted from a crime scene and compared against a large database of fingerprint images. Once the match is found, the camera is determined to be where the “virtual camera” was when the computer image was generated.
One problem with this method is that each time the camera moves, as it is being advanced toward the target, the images recorded by the camera are compared against the large library of computer generated images. This is time consuming and places a strain on the computer resources. It also presents the risk that there may be more than one computer-generated image that closely matches the actual image. For example, if the video camera is up against an airway wall, there may not be much on the image to distinguish it from other similar computer generated images of walls.
Another problem is lack of tracking. Without a sensor, there is no recorded history. Hence, even though the camera is moving and being registered, as soon as the camera encounters an area that matches more than one computer generated image, the registration is lost. The system has no capacity for “tracking” the movement of the camera. In other words, the system does not look at the previous matches to deduce which of the possible images is likely to be the correct one.
Yet another bronchoscope registration method involves terrain or skeletal surface-matching. The virtual model of the lungs is left in a skeletal format, rather than filling the contours in with surfaces and reflections. This saves on initial processing time. As video images are captured of the actual lungs, they are converted into skeletal, digital images. The “real” skeletal images are then matched against the virtual skeletal images. This method requires more processing of the video images than the previously described “virtual surface geometry matching” method but the matching steps are accomplished much more quickly because each of the virtual images is smaller in terms of data. Like the virtual surface matching method, this method present the risk that there may be more than one computer-generated image that closely matches the acquired image, such as when the camera is pointing at a wall.
Each of the aforementioned registration methods has advantages and disadvantages over the others. Generally, the methods using trackable sensors are more accurate than the image-based methods. More particularly, the methods using trackable sensors are more accurate “globally,” that is, they are more accurate when it comes to indicating the present position on a scan of the entire lungs. Image-based methods, on the other hand, can be more accurate “locally,” that is, they can be more accurate relative to a small area, if conditions are optimal. Thus, it would be advantageous to introduce a hybrid method that utilizes the advantages of all of the aforementioned methods.